{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DeepLearning里的 Hello World\n",
    "\n",
    "这是一个综合运用部分，我们会先介绍Conv 层和Pooling 层，然后再进一步介绍最基础的LeNet-5，最后再通过加载MNIST来完成DeepLearning界的 Hello World"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 卷积层（Convlution）\n",
    "\n",
    "### 1.1 为什么需要卷积层？\n",
    "\n",
    "假如我们不采用卷积层而使用向量表示图片，第一、在图片里面相近的像素在向量里面可能会相差很远，很难在空间中表示出它们。 第二、对于大图片输入，模型也会很大，假设我们的图片是256 $\\times$ 256 的， 那么我们的输入层就要有 $ 256 \\times 256 \\times = 196608 $个，如果又1000张图片，那么就会消耗1GB的内存....太大了。\n",
    "\n",
    "### 1.2 什么是卷积层？\n",
    "\n",
    "卷积层跟全连接层类似，只是输入和权重不是做简单的矩阵乘法，而是使用每次作用在一个窗口上的卷积。下图演示了输入是一个$4\\times4$矩阵，使用一个$3\\times3$的权重，计算得到$2\\times2$结果的过程。\n",
    "\n",
    "每次我们采样一个跟权重一样大小的窗口，让它跟权重做按元素的乘法然后相加。通常我们也是用卷积的术语把这个权重叫kernel或者filter。\n",
    "\n",
    "\n",
    "![卷积层](../images/no_padding_no_strides.gif)\n",
    "\n",
    "\n",
    "\n",
    "在PyTorch中，我们可以使用如下代码，完成上述操作\n",
    "\n",
    "\n",
    "另外在PyTorch中，卷积层有 Conv1d、Conv2d、Conv3d， 这三个分别对应着一维卷积、二维卷积、三维卷积，然后它们的输入数据的格式分别是\n",
    "\n",
    "Conv1d的输入数据为 (minibatch, in_chanels, iW)\n",
    "\n",
    "Conv2d的输入数据为 (minibatch, in_chanels, iH, iW)\n",
    "\n",
    "Conv3d的输入数据为 (minibatch, in_chanels, iT, iH, iW)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  0  1  2\n",
      "  3  4  5\n",
      "  6  7  8\n",
      "[torch.FloatTensor of size 1x1x3x3]\n",
      "\n",
      "weights: Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  0  1\n",
      "  2  3\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      "\n",
      "bias Variable containing:\n",
      " 1\n",
      "[torch.FloatTensor of size 1]\n",
      "\n",
      "y: Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  20  26\n",
      "  38  44\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 使用PyTorch的函数来进行卷积操作\n",
    "\n",
    "import torch\n",
    "from torch.autograd import Variable\n",
    "import torch.nn.functional as F\n",
    "\n",
    "x = Variable(torch.Tensor(range(9)))\n",
    "x=x.view(1,1,3,3)\n",
    "print(\"input\", x)\n",
    "\n",
    "weights = Variable(torch.Tensor([0,1,2,3]))\n",
    "\n",
    "weights = weights.view(1,1,2,2)\n",
    "\n",
    "print (\"weights:\",weights)\n",
    "\n",
    "bias = Variable(torch.Tensor([1]))\n",
    "\n",
    "print(\"bias\", bias)\n",
    "\n",
    "y=F.conv2d(x, weights,bias, padding=0)\n",
    "\n",
    "print (\"y:\",y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  0  1  2\n",
      "  3  4  5\n",
      "  6  7  8\n",
      "[torch.FloatTensor of size 1x1x3x3]\n",
      "\n",
      "weights: Parameter containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  0  1\n",
      "  2  3\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      "\n",
      "bias Parameter containing:\n",
      " 1\n",
      "[torch.FloatTensor of size 1]\n",
      "\n",
      "y: Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  20  26\n",
      "  38  44\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 通过 nn 来构建一个卷积模型，来进行卷积操作\n",
    "\n",
    "import torch\n",
    "from torch.autograd import Variable\n",
    "\n",
    "x = Variable(torch.Tensor(range(9)))\n",
    "x = x.view(1,1,3,3)\n",
    "\n",
    "print(\"input\", x)\n",
    "\n",
    "weights = torch.Tensor([0,1,2,3]).view(1,1,2,2)\n",
    "\n",
    "weights = torch.nn.Parameter(weights)\n",
    "\n",
    "print (\"weights:\",weights)\n",
    "\n",
    "bias = torch.nn.Parameter(torch.Tensor([1]))\n",
    "\n",
    "print(\"bias\", bias)\n",
    "\n",
    "# model\n",
    "\n",
    "m = torch.nn.Conv2d(in_channels=x.data.size()[1], out_channels=x.data.size()[1], kernel_size=(2,2), stride=1 )\n",
    "\n",
    "# 设置该卷积层过滤器形状 (out_channels, in_channels, kernel_size[0], kernel_size[1]) 和 权重数值\n",
    "m.weight = weights\n",
    "\n",
    "# 设置该卷积层过滤器偏差\n",
    "m.bias = bias\n",
    "\n",
    "# 求解X\n",
    "y = m(x)\n",
    "\n",
    "print (\"y:\",y)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以控制如何移动窗口，和在边缘的时候如何填充窗口。下图演示了 stride=(2, 2) 和 padding=1。\n",
    "\n",
    "\n",
    "![stride and padding](../images/padding_strides.gif)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  0  1  2\n",
      "  3  4  5\n",
      "  6  7  8\n",
      "[torch.FloatTensor of size 1x1x3x3]\n",
      " \n",
      " \n",
      " weights: Parameter containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  0  1\n",
      "  2  3\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      " \n",
      " \n",
      " bias: Parameter containing:\n",
      " 1\n",
      "[torch.FloatTensor of size 1]\n",
      " y: Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "   1   9\n",
      "  22  44\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y=F.conv2d(x, weights, bias, stride=(2,2), padding=1)\n",
    "\n",
    "print(\"input\", x, \"\\n \\n weights:\" ,weights, \"\\n \\n bias:\", bias, \"y:\", y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当输入数据有多个通道的时候，我们就不能再使用conv2d了，因为它是针对二维数据而设计。\n",
    "\n",
    "但是，我们可以使用conv3d，另外每个通道会有对应的权重，然后可以对每个通道做卷积之后再在通道之间求和\n",
    "\n",
    "\n",
    "$$ conv(data, w, b) = \\sum_i conv(data[:,i,:,:], w[0,i,:,:], b) $$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input Variable containing:\n",
      "(0 ,0 ,0 ,.,.) = \n",
      "   0   1   2\n",
      "   3   4   5\n",
      "   6   7   8\n",
      "\n",
      "(0 ,0 ,1 ,.,.) = \n",
      "   9  10  11\n",
      "  12  13  14\n",
      "  15  16  17\n",
      "[torch.FloatTensor of size 1x1x2x3x3]\n",
      " \n",
      " \n",
      " weights: Parameter containing:\n",
      "(0 ,0 ,.,.) = \n",
      "  0  1\n",
      "  2  3\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      " \n",
      " \n",
      " bias: Parameter containing:\n",
      " 1\n",
      "[torch.FloatTensor of size 1]\n",
      " y: Variable containing:\n",
      "(0 ,0 ,0 ,.,.) = \n",
      "  269  297\n",
      "  353  381\n",
      "[torch.FloatTensor of size 1x1x1x2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# input data\n",
    "x = Variable(torch.Tensor(range(18))).view(1,1,2,3,3)\n",
    "\n",
    "# weight\n",
    "w = Variable(torch.Tensor(range(8))).view(1,1,2,2,2)\n",
    "\n",
    "# bias \n",
    "b = Variable(torch.Tensor([1]))\n",
    "\n",
    "# 计算 y\n",
    "\n",
    "y = F.conv3d(x, w, b)\n",
    "\n",
    "print(\"input\", x, \"\\n \\n weights:\" ,weights, \"\\n \\n bias:\", bias, \"y:\", y)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当然，如果有需要，我们也可以在让输出的数据变成多通道 (由bias控制着)\n",
    "\n",
    "$$ conv(data, w, b)[:,i,:,:] = conv(data, w[i,:,:,:], b[i]) $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input Variable containing:\n",
      "(0 ,0 ,0 ,.,.) = \n",
      "   0   1   2\n",
      "   3   4   5\n",
      "   6   7   8\n",
      "\n",
      "(0 ,0 ,1 ,.,.) = \n",
      "   9  10  11\n",
      "  12  13  14\n",
      "  15  16  17\n",
      "[torch.FloatTensor of size 1x1x2x3x3]\n",
      " \n",
      " \n",
      " weights: Variable containing:\n",
      "(0 ,0 ,0 ,.,.) = \n",
      "   0   1\n",
      "   2   3\n",
      "\n",
      "(0 ,0 ,1 ,.,.) = \n",
      "   4   5\n",
      "   6   7\n",
      "\n",
      "(1 ,0 ,0 ,.,.) = \n",
      "   8   9\n",
      "  10  11\n",
      "\n",
      "(1 ,0 ,1 ,.,.) = \n",
      "  12  13\n",
      "  14  15\n",
      "[torch.FloatTensor of size 2x1x2x2x2]\n",
      " \n",
      " \n",
      " bias: Variable containing:\n",
      " 1\n",
      " 1\n",
      "[torch.FloatTensor of size 2]\n",
      " y: Variable containing:\n",
      "(0 ,0 ,0 ,.,.) = \n",
      "   269   297\n",
      "   353   381\n",
      "\n",
      "(0 ,1 ,0 ,.,.) = \n",
      "   685   777\n",
      "   961  1053\n",
      "[torch.FloatTensor of size 1x2x1x2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# input data\n",
    "x = Variable(torch.Tensor(range(18))).view(1,1,2,3,3)\n",
    "\n",
    "# weight\n",
    "w = Variable(torch.Tensor(range(16))).view(2,1,2,2,2)\n",
    "\n",
    "# bias \n",
    "b = Variable(torch.Tensor([1, 1]))\n",
    "\n",
    "# 计算 y\n",
    "\n",
    "y = F.conv3d(x, w, b)\n",
    "\n",
    "print(\"input\", x, \"\\n \\n weights:\" ,w, \"\\n \\n bias:\", b, \"y:\", y)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二、池化层（pooling）\n",
    "\n",
    "\n",
    "因为卷积层每次作用在一个窗口，它对位置很敏感。池化层能够很好的缓解这个问题。它跟卷积类似每次看一个小窗口，然后选出窗口里面最大的元素，或者平均元素作为输出。\n",
    "\n",
    "\n",
    "![池化层](../images/Max_pooling.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "跟上面的卷积层一样，我们先使用 torch.nn.functional 来进行池化操作，然后再使用 torch.nn来在模型中构建池化层"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "   0   1   2   3\n",
      "   4   5   6   7\n",
      "   8   9  10  11\n",
      "  12  13  14  15\n",
      "[torch.FloatTensor of size 1x1x4x4]\n",
      " \n",
      "  kenel_size: (2, 2)  \n",
      "  y_Max: Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "   5   7\n",
      "  13  15\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      " \n",
      " y_ave: Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "   2.5000   4.5000\n",
      "  10.5000  12.5000\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.nn.functional as F\n",
    "from torch.autograd import Variable\n",
    "# input data\n",
    "x = Variable(torch.Tensor(range(16))).view(1, 1, 4, 4)\n",
    "\n",
    "# kenel_size\n",
    "\n",
    "kenel_size = (2, 2)\n",
    "\n",
    "# Max-pooling\n",
    "y_Max = F.max_pool2d(x, kenel_size)\n",
    "\n",
    "# average-pooling\n",
    "y_ave = F.avg_pool2d(x, kenel_size)\n",
    "\n",
    "print(\"input\", x, \"\\n  kenel_size:\" ,kenel_size,  \" \\n \", \"y_Max:\", y_Max, \"\\n y_ave:\", y_ave)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们使用 torch.nn 来构建模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "   0   1   2   3\n",
      "   4   5   6   7\n",
      "   8   9  10  11\n",
      "  12  13  14  15\n",
      "[torch.FloatTensor of size 1x1x4x4]\n",
      " \n",
      "  kenel_size: (2, 2)  \n",
      "  y_Max: Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "   5   7\n",
      "  13  15\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      " \n",
      " y_ave: Variable containing:\n",
      "(0 ,0 ,.,.) = \n",
      "   2.5000   4.5000\n",
      "  10.5000  12.5000\n",
      "[torch.FloatTensor of size 1x1x2x2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import torch.nn as nn\n",
    "\n",
    "x = Variable(torch.Tensor(range(16))).view(1, 1, 4, 4)\n",
    "\n",
    "# kenel_size\n",
    "\n",
    "kenel_size = (2, 2)\n",
    "\n",
    "\n",
    "m_max = nn.MaxPool2d(kenel_size)\n",
    "\n",
    "m_ave = nn.AvgPool2d(kenel_size)\n",
    "\n",
    "\n",
    "# 求解\n",
    "\n",
    "y_Max = m_max(x)\n",
    "\n",
    "y_ave = m_ave(x)\n",
    "\n",
    "print(\"input\", x, \"\\n  kenel_size:\" ,kenel_size,  \" \\n \", \"y_Max:\", y_Max, \"\\n y_ave:\", y_ave)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 三、LeNet-5 模型\n",
    "\n",
    "接下来我们会使用LeNet模型来处理MNIST数据集。\n",
    "\n",
    "LeNet-5的模型结构如下图所示：\n",
    "\n",
    "![LeNet-5](../images/LeNet-5.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 构建模型\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "from torch.autograd import Variable\n",
    "import torch.nn.functional as F\n",
    "\n",
    "\n",
    "class LeNet5(nn.Module):\n",
    "    def __init__(self, in_dim, n_class):\n",
    "        super(LeNet5, self).__init__()\n",
    "        # 从结构图中可以看出，第一层：卷积层输入是1 channel, 输出是 6 channel, kennel_size = (5,5)\n",
    "        self.conv1 = nn.Conv2d(in_dim, 6, 5, padding=2)\n",
    "        # 第二层：依旧是 卷积层， 输入 6 channel 输出 6 channel , kennel_size = (5,5)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        # 第三层：全连接层（线性表示）\n",
    "        self.fc1 = nn.Linear(16*5*5, 120)\n",
    "        # 第四层：全连接层\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        # 第五层：输出层\n",
    "        self.fc3 = nn.Linear(84, n_class)\n",
    "    # 向前传播\n",
    "    def forward(self, x):\n",
    "        # Subsampling 1 process\n",
    "        x = F.max_pool2d(F.relu(self.conv1(x)), 2)\n",
    "        \n",
    "        # Subsampling 2 process\n",
    "        x = F.max_pool2d(F.relu(self.conv2(x)), 2)\n",
    "        \n",
    "        # -1的话，意味着最后的相乘为维数\n",
    "        x = x.view(-1, self.num_flat_features(x))\n",
    "        # full connect 1\n",
    "        x = F.relu(self.fc1(x))\n",
    "        # full connect 2\n",
    "        x = F.relu(self.fc2(x))\n",
    "        # full connect 3\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "    \n",
    "    # 6 channel 卷积层 转全连接层的处理\n",
    "    def num_flat_features(self, x):\n",
    "        # 得到 channel * iW * iH 的值\n",
    "        size = x.size()[1:]\n",
    "        num_features = 1\n",
    "        for s in size:\n",
    "            num_features *= s\n",
    "        return num_features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LeNet5 (\n",
      "  (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n",
      "  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (fc1): Linear (400 -> 120)\n",
      "  (fc2): Linear (120 -> 84)\n",
      "  (fc3): Linear (84 -> 10)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "leNet = LeNet5(1, 10)\n",
    "print(leNet)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集是 MNIST dataset\n",
    "\n",
    "我们会使用 torchvision 来加载数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torchvision\n",
    "from torch.utils.data import DataLoader\n",
    "from torchvision import transforms\n",
    "from torchvision import datasets\n",
    "\n",
    "# mini-batch\n",
    "batch_size = 128\n",
    "\n",
    "# 未下载数据，使用True表示下载数据\n",
    "DOWNLOAD = False \n",
    "\n",
    "train_dataset = datasets.MNIST(\n",
    "    root='./data', train=True, transform=transforms.ToTensor(), download=DOWNLOAD)\n",
    "\n",
    "test_dataset = datasets.MNIST(\n",
    "    root='./data', train=False, transform=transforms.ToTensor())\n",
    "\n",
    "train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\n",
    "test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们有了模型，也有了数据集，就让我们开始进行测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1\n",
      "**********\n",
      "[1/2] Loss: 1.630987, Acc: 0.528724\n",
      "epoch 2\n",
      "**********\n",
      "[2/2] Loss: 0.439152, Acc: 0.871667\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "# hyper-parameters\n",
    "learning_rate = 0.0001\n",
    "num_epoches = 2\n",
    "use_gpu = torch.cuda.is_available()\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.Adam(leNet.parameters(), lr = learning_rate)\n",
    "tt = 0\n",
    "# 开始训练\n",
    "for epoch in range(num_epoches):\n",
    "    print('epoch {}'.format(epoch + 1))\n",
    "    print('*' * 10)\n",
    "    running_loss = 0.0\n",
    "    running_acc = 0.0\n",
    "    for i, data in enumerate(train_loader, 1):\n",
    "        tt +=1\n",
    "        img, label = data\n",
    "        img = Variable(img)\n",
    "        label = Variable(label)\n",
    "        # 向前传播\n",
    "        out = leNet(img)\n",
    "        loss = criterion(out, label)\n",
    "        running_loss += loss.data[0] * label.size(0)\n",
    "        _, pred = torch.max(out, 1)\n",
    "        num_correct = (pred == label).sum()\n",
    "        accuracy = (pred == label).float().mean()\n",
    "        running_acc += num_correct.data[0]\n",
    "        # 向后传播\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        if i % 300 == 0:\n",
    "            print('[{}/{}] Loss: {:.6f}, Acc: {:.6f}'.format(\n",
    "                epoch + 1, num_epoches, running_loss / (batch_size * i),\n",
    "                running_acc / (batch_size * i)))\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Baseline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
